Generalized Linear Models

Basics of Generalized Linear Models (GLMs)

• GLMs are a flexible generalization of ordinary linear regression that allows for response variables that have error distribution models other than a normal distribution.
• Components of GLMs:
  • Random Component: Specifies the distribution of the response variable (e.g., Normal, Binomial, Poisson).
  • Systematic Component: A linear predictor, a combination of explanatory variables (predictors).
  • Link Function: Connects the mean of the response variable to the linear predictor (e.g., identity, log, logit).

Common link functions and GLMS

• Identity Link: Used for normally distributed data (linear regression).
• Logit Link: Used for binary outcome data (logistic regression).
• Log Link: Used for count data (Poisson regression).
• Reciprocal Link: Used for increasing rate that levels off (Gamma regression).

Over-Dispersed Data

• Over-dispersion in general refers to having variance greater than that assumed for the theoretical data model.
  
• Over-dispersion can also refer to having a variance that is greater than the mean
  • Similarly equa-dispersion would refer to having variance equal to the mean.

Common applicataions

Causes

Candidate Distributions

1. Negative-Binomial
2. Generalized Poisson
3. Double Poisson
4. Conway-Maxwell-Poisson (CMP)

• Zero inflated distributions
  • ZIP
  • ZINB
  • ZIDP/ZIGP

Negative-Binomial

• Parameters: mean: \(\mu\), dispersion: \(k\)¹
• Variance: \(\mu + \mu^2/k\)
  • Function of mean and dispersion parameter
  • Clearly captures over-dispersion.

Generalized Poisson

• Parameters: \(\lambda\), \(\theta\)
• Mean: \(\lambda / (1- \theta) \quad\) Variance: \(\lambda / (1- \theta)^3\)
• Model introduced by Consul 1989 as a way to modify the Poisson to handle over-dispersed and under-dispersed count data
• Probability distribution function \[ Pr(Y = y) = \frac{\lambda(\lambda + \theta y)^{y-1} e^{-(\lambda + \theta y)}}{y!}, \quad \lambda > 0,\space \theta \in \mathbb{R} \]

Double Poisson

• Parameters: \(\mu\), \(\theta\)
• Mean: \(\mu \quad\) Variance:¹ \(\mu / \theta\)
• Extension of the double exponential family (Efron 1986) with approximate pmf \[ Pr(Y = y) = (\theta^{1/2}e^{-\theta\mu}) \left(\frac{e^{y}y^y}{y!}\right) \left(\frac{e \mu}{y}\right)^{\theta y} \]
• The exact double Poisson (DP) density includes a normalizing constant \(\sum_{y=0}^{\infty} Pr(Y = y) \approx 1 + \frac{1 - \theta}{12\mu\theta}(1 + \frac{1}{\mu\theta})\)

Conway-Maxwell Poisson (CMP)

• Parameters:¹ \(\lambda\), \(\nu\)
• Mean: \(\mu \approx \lambda + 1/2\nu - 1/2 \quad\) Variance:² \(\sigma^2 \approx \lambda / \nu\)
• Weighted Poison distribution with pmf: \[ Pr(Y = y) = \frac{\lambda^y}{(y!)^\nu Z(\lambda, \nu)}, \quad Z(\lambda, \nu) = \sum_{y=0}^\infty \frac{\lambda^y}{(y!)^\nu} \]
• Includes spacial cases (Sellers et al. 2012) of Poison when \(\nu = 1\), geometric when \(\nu \to 0\), and Bernoulli when \(\nu \to \infty\)

Zero Inflated Distributions

• Zero inflated distributions are piece-wise distributions with components for 0s and non-0s
• For example the zero inflated Poisson (ZIP) has pmf \[ Pr(Y = y) = \cases{ \pi + (1-\pi)e^{-\lambda} &\text{if } y = 0\\ (1 - \pi)\frac{\lambda^y e^{-\lambda}}{y!} &\text{if } y = 1, 2, \ldots } \]
• Zero inflated framework extends to other distributions to further capture over-dispersion and zero inflation
• Models can be fit using the zeroinfl() function in the package pscl

Results

• It has been found and shown that modeling over-dispersed data with improper distributions leads to biased results.
• To prevent biased results from over dispersed data, using models such as the CMP, GP, or DP model can prove beneficial
• Zero inflated models have better fit when there are increased number of zeros observed and can easily be implemented
• Some of these models are easy to implement such as the CMP and CMP(mean-parameterized) in packages COMPoissonReg and mpcmp